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ABSTRACT 

Two studies were conducted to examine the efficacy of 
direct measurement, standardized achievement tests, and 
aptitude-achievement discrepancy scores in distinguishing learning 
disabled (LD) and, nonlearning disabled (NLD) students in grades 3 to 
6, For both reading (Study I) and written expression (Study II ), 
students 1 scores on ,direct and repeated measures (Words in Isolation' 
and Oral Reading) predicted their classification as LD or NLD as well 
as commercial measures of achievement (Stanford Diagnostic Reading 
Test, the Peabody Individual Achievement Test, and the Test of 
Written Language) and aptitude-achievement discrepancy (the 
Woodcock-\Johnson Psycho-Educational Battery). However, the direct 
measures required significantly smaller expenditures, both in terms 
of cost and time. The additional benefit was the use of direct, 
repeated measures to obtain common data bases across decisions 
affording continuity between information collected during assessment 
and the information needed to evaluate instruction. (Author/PN) 
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Abstract 

Two studies were conducted to examine the efficacy of direct 
measurement, standardized achievement tests, and aptitude-achievement 
discrepancy scores in distinguishing learning disabled (LD) and non- 
learning disabled (NLD) students. For both reading (Study I) and 
written expression (Study II), students' scores on direct ..and repeated 
measures predicted their classification as LD or NLD as well as 
commercial measures of achievement and aptitude-achievement 
discrepancy. However, the direct measures required significantly 
smaller expenditures, both in terms of cost and time. The additional 
benefit of the use of direct, .repeated measures to obta.in common data 
bases across decisions also is discussed. 
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Predictive Efficiency of Direct,' Repeated Measurement: 
An Analysis of Cost and Accuracy in Classification ^ * ' > 

At a recent national symposium, participants expressed 
substantial concern about the ambunt 'of testing done by school 
psychologists (Peterson, 1981). Indeed, in a recent survey, school 
psychologists estimated that they spend about 70% of their time 
engaged in assessment activities (Goldwasser, ^Meyers, Christenson, & 
Graden, 1982). The rate* is alarming given the many limitations of 
test-based assessment.' Arter and Jenkins (1?79), for example, found 
that most tests measuring perceptual-motor .skills and psychol inguistic 
functioning possessed low, validity and reliability. In addition, they 
found lictle evidence to support the use of the tests in developing 
ren^dial programs for students with handicapping conditions. 

Problems with test-based assessment do no't ,exist only in 
perceptual-motor and psycholinguistic tests. Use of popular 
standardized tests of intelligence and achievement for assessment also 
has been criticized because of the poor technical adequacy K of many 
devices (Salvia & Ysseldyke, 1981). Diagnosing learning difficulties 
on the 'basis of discrepancy scores from these tests ' is also 
problematic (Salvia & Clark, 1973; Algozzine, Ysseldyke, & Shinn, in 
press). - Perhaps the most provocative data come from surveys of how 
school psychologists and teachers ,view the instructional usefulness of 
standardized tests (Thurlow &< Ysseldyke, 19821.. Approximately 80% of 
surveyed school psychologists reported that they bel ieve the 
standardized tests they administer are educationally relevant. 
However, only 30% of surveyed teachers indicated that these measures 
give them- informationAiseful in^ their instructional -planning-; 
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As a result of these assessment-related -problems, school 
psychologists find their activities criticized. Keogh (1972) observed 
that the school psychologists 1 diagnostic contribution was limited, 
and argued that "a somewhat different model of school psychology be 
adopted ?f services to exceptional children are to be effective 11 (p. 
144). ' 

One possible alternative model is the use of direct,, reputed 
measurement of academic skills (Lovitt, 1967; White & Haring, 1980)., 
The advantages of this methodology are two-fold. First, the data are 
related directly to instruction, a qualnty that many standardized 
tests do not possess (Bersoff, 1973; Goldstein, Arkell, Ashcroft, 
Hurley, & Lilly, 1975). Second, the measures are time efficient and 
may be administered by both school psychologists and teachers. This 
is an important characteristic considering the amount of time 
psychologists are using for assessment. Tr^ctjnan (1981), for example, 
talks about a role change for the school psychologist, where he or she 
is an "enabler"* who moves beyond the role of tester and wprks % closely 
with teachers in the classroom. New concepts such as these, however, 
will be difficult to implement given the previously cited 70# of 
school psychologist time devoted to testing. k 

While direct, continuous measurement originally was developed ^or 
measuring pupil progress, it may be useful in making student placement 
decisions (Jenkins, Deno, & Mirk in^ 1979). Mirkin, Marston, and Deno 
(1982) found that students referred by direct measurement procedures 
and students referred by teachers did not differ in performance on 
standardized tests of intelligence and achievement. Shinn, Ysseldyke, 



Deno, and Tinda.l (1982) demonstrated that standardised tests did not 
significantly discriminate between learning disabled and low-achieving 
students whereas direct measurement of academic skills did 
successfully differentiate ttye two groups. 

The purpose of the research presented here was to continue the 
comparison of the util ity of traditional^, commercially * available 
standardized tests with direct measurement. For the academic areas of 
reading and written expression, discriminant analysis techniques were 
used to investigate the efficacy of both approaches to classifying 
students accurately as learning disabled (LD) or not learning disabled 
(NLD) . Specifically, canonical correlations and classification 
coefficients derived from discriminant analysis were used to compare 
the direct measurement procedures with traditional measurement 
procedures* 

Two studies were conducted, one in the area of reading and one in 
the area of written language. In each study, the efficacy of direct 
measurement, standardized tests of* achievement, and* aptitude- 
achievement discrepancy scores in distinguishing learning disabled and 
non-learning disabled students was examined by discriminant analyses. 

Study I - Reading • A 

• Study I contrasted the performance of LD and NLD students on 
direct measures and* standardized 'tests of reading. 
Method • J 

Subjects . Forty-three (M = 22, F = 21) regular program students 
and 23 (M = 19, F = 4) LD students participated in this study. The 
grade placement level of these 66 children ranged from first through 



sixth grades. Their ages ranged from 78' months to 156 months, with a 
median of 112 months. The subjects were from three Minneapolis inner 
. city schools. 

In order .to .examine how well aptitude-achievement difference 

scorer, In reading discriminated between LQ and NLD groups, a second 

*• -I » 

sample of 65 students was studied. These students were enrol led § in 
^grades .3-6 in three Midwestern elementary schools. All students had 
been referred for special education; ?5 were placed in L'^ services. 
The remaining 40 students were not eligible for special education 
services; these students made up the non-learning disabled (NLD) 
group. 

• * *■" # 

Standardized commercial tests . The achievement measures in 

reading included two published tests, the Stanford, Diagnostic Reading 

Test (SDRT; Karlsen, Madden, & Gardner, 1975), Subtest Five, Part A 

(Reading Comprehension and Phonetic Analysis) of Form B and the 

Reading Comprehension subtest from the Peabody Individual Achievement 

Test (PIAT; Dunn & Markwardt, 1970) . The aptitude-achievement 
— —— — — ^ 

discrepancy scores for reading were derived from scores obtained on 
the Woodcock -Johnson Psycho-EducationaJ Battery (Woodcock & Johnson, 
i.978). 

Direct measures of reading . Two direct measures of reading were 

used. The Words in. Isolation measure consisted of 'three alternative 

/ • 

forms of 60 words each that were randomly selected by grade level from 
the Core List of ^ 5,167 words listed in Bas i c El emen tary T-Re'ad i nq 
Vocabulary (Harris & Jacobson, 19/2). Each 60-word list consisted of 
10 words from each of the six grade levels. Words were included on 



the word o lists only if they had a frequency index of^more than 10 per 
million words in th.e Teacher's Wordbook of 10,000 Words (Thorndike & 
Lorge, 1944). After a pool^of 60 words was .obtained -tor each list, 

« "** . * 

s, < «• 
«■ * 

the words, were typed in 12 rows with five words 'in each row. For-the 

Words in Isolation measure the mean number of words 'read correctly in 

one minute was the dependent variable. „ <• 

« 

The Oral Reading measure . included three passages of 300^words 
each. These were selected^from three third grade basal readers,' 
(Allyn-Bacon, Ginn 720, Houghton-Miff 1 in) and each was typed on a 
sheet of paper. Each passage consisted" of tfie first part of a story.* 
The reading levels for the passages were computed uSing the Fry 
Readability Index formula (Fry, t968) and each was at the third grade 
level. For this measure, the mean number of words read correct in one 
minute was the dependent variable. 
Results . * 

The mean reading performance on the various, measures for the LD 
and NLD groups is presented in Table 1. The means^of the LD students 
were lower. than those of the NLD students on both the direct measures 
and the commercial tests.' The discrepancy scores of the LD students 
and the NLD students were similar. 

----- ----------------------- - „ 

Insert Table 1 about here 



e The results* of the discriminant analysis are shown in Table 2. 
For the direct measures of reading, the number of words read correctly 
from the word lists and basal passages were entered into the analysis. 



The canonical correlation, .between the "direct measures and v bcT-NLD, 

• • • 

criterion was .&3. # 'Using *the derived discriminant filnctiop, 83% of 

all cases 'could* be correctly classified. For the standardized test§ 

of reading achievement (SDRT apd PIAT), the canonical* correlation was 

.70, with 86# of the cases correctly classified. The Woodcodk-Jdhnson 

Aptitude-Achievement .Discrepancy score for reading, diti« not correlate 

highly^ih the gritjerjon; the canonical correlation was <.07. Thus, 

* * 

cases correctly classified as LD or. NLD by discrepancy scores was -only 
62%. * \ • * '* 



Insert Table 2 about here 



A further breakdown of.LD'and NLD students classified correctly 

* * « 

and incorrectly is. presented in Ta*ble 3.. Hit rates and percentages of 
*false positives were similar for the** direct measures and the 
standardized achievement tests. The discrepancy scores for reading, 
however, provided^ little information for 'classif jcatj on purposes. 



•Insert Table 3 about here 



Study II - Written Expression ■ > 

l< • * , . ** ' « 

Study II compared, the classification of LD and NLD students- with 

• ? f 

performance on direct , measures and standardized tests of written 
language. 

Subjects • 

Eighty-two children were selected randomly from five elementary ^ 



schools in the Twin Cities area.. The 42 males and 40 females were in 
grades 3-6 and ranged in age from 7 to 11 years old. Thirty-one of 
these-children were in learning disability resource programs. 

Subjects included in the examination of the aptitude-achievement 
discrepancies for written language were the same as those included in 
the reading discrepancy analysis. 

Standard i zed commer c i a 1 tes t s . The Test of Written Language 
(TOWL), developed by Hammill and Larsen (1978), was used to measure 
written expression. The TOWL consists of five subtests: Vocabulary, 

Thematic Maturity, Spelling, Word Usage, and Style. The scores 

- ' * c 

obtained from the TOWL and used for analysis were scaled scores. 

Direct measures , of 'written expression . Following guidelines set 

up by Deno, Marston, and^Mirkin (1982), each student was asked, to 

write a composition after ttfey were presented a story starter. At the 

end of five minutes, each composition was scored for Total Words 

,Written,' Words Spelled Correctly, and, Correct Letter Sequences (^hite 

(& Haring, 1980). ' > 

! Results* . *• • 

Means and standard deviations for the written language measures 

v * 

are presented in TaDle 4. On all direct measures and published 
measures* of achievement, the mean score of LD. Students was lower than 
that of NLD* students. The mean discrepancy score was greater for LD 

: 

students than for NLD students. 



Insert Table 4 about here 



Discriminant analysis was used to examine combinations of direct 
measures and standardized achievement tests \n L predicting group 
membership. The$e results are shown in Table 5. Foxthe direct. 

measures, of written expression, Total Words, Words Spelled Correct, 

• * \ 
and Correct Letter Sequences were entered, into the analysis. ^The 

resulting discriminant function produced a canonical correlation ^of 

.52, with 73% of the cases correctly classified. For the standardized 

achievement tests measuring written expression the subtests of the 

Test of Written Language were enterec} into the discriminant analysis. 

the resulting function correlated .68 with .the criterion and correctly 

classified 80% of the cases. Investigation of^.the predictive 

efficiency of discrepancy scores focused on the -Woodcock-Johnson 

Aptitude-Achievement difference scores. Canonical correlation for 

this index^was .43; with 74% of the "students placed correctly. 



Insert Table 5 about here 



A more" detail ed^ analysis of the percentage of correct 
classifications is presented in Table 6. Hit rates ranged from 43% to 



93%, while false positives^ ranged from 7% to 57%. In contrast to 
re*ading t discrepancy scores, c the written language expression 
discrepancy scpres were similar in their accuracy of classification to 
,both th'e direct measures and the standardized tests. 



Insert Table 6 about here 



Discussion 

Direct and- repeated measurement procedures initially were 
designed for use in .monitoring student progress in resource programs 
(Jenkins et al., 1979). However, eduqators face numerous assessment 
decisions in addition to program monitoring: screening, 
identification, instructional planning, and .exit from service. 
Typically, the data gathered for these other decision areas come from 
commercially available standardized tests. The research presented 
here supp.orts the notion that direct and repeated measurement also may 
be used for making screening and identification decisions. In our 
attempt to differentiate school -class if ied LD and NLD students in 
reading and written expression, we found that direct, repeated 
measurement predicted* student eligibility as effectively as commercial 
measures of achievement and aptitude-achievement discrepancy. 

Given that direct, repeated measurement is the most satisfactory 
method of measuring student progress on IEP goals, the benefit of 
„ using the same assessment approach for screening, identification, and 
exit criteria becomes apparent. First, the difference in cost and 
efficiency of making these decisions among" the various approaches is 
significant. The materials used for direct, repeated measurement were 
produced at minimal cost # and required only a few minutes to 
administer. On the other hand, the published tests used were all 
quite expensive and required a far greater amount of time to 
administer. For example, at least one hour is required to administer 
the PIAT and over two hours is needed to administer the aptitude and 
achievement sections of the Woodcock- Johnson Psycho-Educational 
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Battery. Thus, the notion that direct and repeated measures have 
"predictive efficiency" is suggested. 

A second, and more important, advantage of direct, repeated 
measures is the development of a common data base across decision 
areas. With the use of direct, repeated measures from the curriculum, 
there is continuity between the information collected during 
assessment and the information needed to evaluate instruction. 

As Salvia and Ysseldyke (1981) pointed out, assessment is 
essentially the collection of data for the* purpose of making 
decisions. The five decisions that they describe include 
screening/referral, eligibility, program planning, program evaluation, 
and outcome evaluation. The use of informal, curriculum-based 
measures during initial assessments in screening and eligibility 
avoids the inherent discontinuity in the data base that results when 
published, commercial tests are t used. In c this latter system, one type 
of information is collected initially, and then during instruction 
(program planning" and evaluation), another type of. information is 
generated and collected. Further, outcome evaluation using published 
tests is determined in a manner that is receiving increasing criticism 
(Jenkins^ & Pany, 1978; Skager, 1980). Finally, shifts in the 
measurement system preclude any meaningful analysis of the effects of 
educational decisions that were made. 

In a related manner, the use of informal curriculum-based 
measures results in a system of assessment capable of adapting into a 
program of continuous evaluation. Because the materials involve 
minimal additional costs, are capable of providing multiple alternate 



11 

forms, and are time efficient to administer, the system can be 
implemented on a frequent, if not daily, basis. That is, assessment 



need not stop once instruction begins, but may continue throughout the 
entire educational program of the student. 



< 
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Table 1 

Means and Standard Deviations on 
Reading Measures for LD and NLD Students 



Learning Disabled Non-Learning Disabled 
Reading Measures X S.D. X S.D. 

Direct Measures 

■ * Word List - Words . t - 1,2.2 16.2 53.3 26.4 

Read Correctly 

' Oral Passage - Words 39.8 26.4 9.4.0 33.5 

Read. Correctly 

Comroer&ia.l Achievement Tests- * 

^--^ 

Peabody Individual 24.5 14.9 60.7 23,4 

Achievement Test"" ^ * 
(Total Percentile) 7 _ 

Stanford Diagnostic 15.0 13.2* / _ 51 . 5 27.4 

Reading Test ^ 



r\cau i ii y icji. ^ 

-.(Jota-t Percent i-1 e> 



Discrepancy Scores 

Woodcock- Johnson -&.6 15.1 -6.8 10.5 

Reading Aptitude - 
Reading Achievement 
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. < / Table 2 
Canonical Correlations and Classification Coefficients 
Derived from Discriminant Analyses 



Reading Measures " 



Canonical 
Correlations 



Percentage 

Correctly 

Classified 



Direct Measures 

. (Word List and Oral Passage) 

Commercial Tests 



Achievemen treasures 
(PIAT and SDRT) 

Discrepancy Score * 
(Woodcock-Johnson), 



.63 



.70 



.07 



83 



86 



62 



v 



V 



X 
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Table 3. 

•Number and Percentage of Actual LD and NLD Students 
Who Were Classified With Discriminant Analysis 



Actual firoup 
Classification. 



Predicted Group' Classification 



Number and 
Percentage of 
Students Class- 
ified as LD , 



Number and 
Percentage of 
Students Class- 
ified as NLD 



Direct Measures 

(Word List arid OVal Passage) 

Acttial number of students 

classified as LD (N=14) 



Actual number of. students 
' classified as NLD (N=21). 

Commercial Achievement Jests 
(PIAT and SDRT) . 

Actual number of students 
classified as LD (N=14) 

Actual number of students 
classified as NLD (N=21) 

Commercial Discrepancy Score 
(Woodcock-Johnson - Reading) 

Actual number of students 
classified as LD (N=25) 



Actual number of students 
classified as NLD (N=40) 



11 31%) 



3 (14%), 



13* (.93%) 
4 (19%) 



0 (0%) 
0 (0%) 



3 (19%) 



18 (86%) 



1 (7%) 
17 (81%) 



25 (100%) 
40 (100%) 



Table 4 

Means and Standard Deviations on Written Expression Measures 

for LD and NLD Students 



learning Disabled NorT-Learning Disabled 
Written Expression Measures X S.D. X S.D. 



Direct Measures 

Total Words Written ' 37.2 

Words Spelled Correctly " 30.8 

Correct Letter Sequences 144.9 

Commercial Achievement Tests 



Test of Written Language 
(Scaled- Subtest Scores) 



Vocabulary 7.3 

Thematic Maturity 7.-0 

Spelling 5.2 

Word Usage 5.1 

\ Style 5.1 

Discrepancy Scores 

Woodcock- Johnson - -15.6 
Written Language 
Ap,titude",Ach j evemen t , 



21.6 


58.4 „ 


20.0 


21.1 


53.4 


20.5 


94.3 


238.5 


91 .6 


4.3 


11.2 


3.4 


2.0 


9.9 


2.8 


2.7 


10.3 


3._6 


2.6 


10.9 


3.6" 


1.6 


• 9.. 4 


3.-4 


15.2 


r 

-4.2 


9.5 



Table 5 

Canonical Correlations and Classification Coefficients 
Derived from Discriminant Analyses 



0 

Written Expression Measures 


Canonical 
Correlations 


Percentage 

Correctly 

Classified 


3 . 

Direct Measures 
(Total Words, Words Spelled 
Correctly and Correct 
Letter Sequences) . 


.52 


c 


Commercial Tests 






■ Achievement Measure 

(TOWL: All 5 subtests) 


.68 

t 


80 


Discrepancy Score 
(Woodcock-Johnson) 


» 

.43 


74 
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Table 6 

Number and Percentage of Actual LO and NLD Students 
Who Here Classified with Discriminant Analysis 



Actual ' oup 
Classification 



o Predicted Group Classification , 
Number and Number and . 

Percentage of Percentage of 
Students Class- Students Class- 
ified as LD ified as NLD 



Direct Measures v 

(Total Words, Words Spelled 
Correctly, Correct Letter * 
Sequences) 

Actual number of stucTents 14 (52%) 

classified as LD (N=27) 

Actual number of students 6 (14%) 
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